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Abstract 


In some important image coding techniques such as transform coding, an image is divided 
into several no of blocks and then each block is coded mdepedently One of the drawbacks 
of Discrete Cosine Transform (DCT) is visible block boundaries due to coarse quantization 
of the coefficients In this thesis an adaptive image restoration filter using DCT-based edge 
classification has been proposed for reducmg block artifacts m compressed images Edge 
direction of each block is classified by using DCT coefficients and an adaptive constrained 
least square (CLS) filter along the edge direction is used for filtering the corresponding 
block Computer simulations have been used to evaluate the performance of the proposed 
restoration filtering technique 


Chapter 1 


Introduction 


Digital image processing refers to processing of a two-dimensional picture by a digital 
computer Representing an image m digital form requires a large no of bits With the rapidly 
growing applications the total volume of digital images produced each day is increasing As 
a result, it has become necessary to find efficient representation for digital images in order to 
reduce the no of bits, bandwith of the the transmitting channel, etc Digital image coding is 
used m a variety of contexts, including conservation of bandwith in transmitting images and 
conservation of memory space in stonng images The objective of image coding is to represent 
the image with as few bits as possible while retaining the sufficient picture information in 
reconstruction For effiecient transmission of digital networks, digital video needs to be 
compressed to meet the bandwidth constraints As the demand for video communication has 
grown, many effiecient image compression techniques have been developed and standardized 
Especially, good quality image communication with low bit-rate is gaming growing interests 
in applications to video conferencing, videophone, interactive TV etc When images are 
compressed at low bit-rate, images with more details usually degrade more than those with 
fewer details The original image is segmented into blocks, usually of equal size and then 
each block is coded independently Discrete Cosine Transform (DCT) is among the most 
popular transform technique for image processing because of its significant energy compaction 
property [11, 13] and ease of implementation The mam drawback of the block discrete cosine 
transform (BDCT) based compression technique is blocking artifact [2] which results from the 
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independent processing of the blocks without taking into account the block pixel correlations 
and produces artificial discontinuity between adjacent blocks(fig 5 2) The lower the bit- 
rate, the more severe the blocking artifacts due to loss of information Decoded images 
exhibiting blocking effects can be unpleasant to a viewer As the bit rate is reduced, the 
blocking effect, which is not noticeable at high bit rates, becomes more prommant 

Ther are many methods available for reducing the blocking artifacts For example, in the 
overlap method abrupt discontinuities caused by coding can be reduced because the segmen- 
tation process weaves the adjacent subimages together[l] The overlap method reduces the 
blocking effects well without degrading the image content The major disadvantage of this 
method is increase in bit-rate based on images In the iterative image restoration methods 
the image quality can be improved significantly with sufficient number of iterations[3,5,6] 
But because of its slow convergence this method is not suitable for real-time processing 
Even m JPEG slow convergence is definitely a disadvantage 

Here a post-processing technique has been used to reduce the blocking artifacts In 
order to reduce blocking artifacts effieciently, edge direction of each block is classified by 
using DCT co-efficients and an adaptive constrained least square (CLS) filter along the edge 
direction is used for filtering the corresponding block For real-time application, the proposed 
restoration filter is implemented m the form of a truncated FIR filter, which is suitable for 
postprocessing the images in real time video systems such as HDTV or video conferencing 
The thesis is organised m the following way Chapter 2 gives the degradation model 
for the blocking artifacts Chapter 3 gives the adaptive image restoration based on the 
degradation model descnbed in chapter 2 FIR implementation of the proposed algorithms 
for real-time processing is descnbed m chapter 4 Finally simulation results and conclusions 
are given in chapter 5 and 6 respectively 
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Chapter 2 


Degradation Model for Block Artifact 


In digital image processing a two dimensional image is segmented into blocks, usually of 
equal size and each block is coded independently The purpose of image coding is to represent 
the image with as few bits as possible while retaining the sufficient picture information 
Here an NxN image x(m,n) is divided into subimages of size BxB called blocks One 
dimensional representation of an NxN image x(m,n) for BxB block based processing, 

x = [xjx%, ,x t nVB 2 ) t (21) 

where x * for k = (p—l)(N/B) +q represents the lexicographically ordered B 2 elements in 
the (p, q)th block Block diagram of the image degradation for the BDCT-based compression- 
reconstruction process is shown in fig 2 1 


degradation phase based on BDCT restoration phase 



Figure 2 1 Block diagram of degradation restoration model for the BDCT based 
compression-reconstruction process 
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The degradation model can be expressed as 


y = C~ 1 D~ 1 QCx 


(2 2 ) 


where y represents the reconstructed image with block axtifacts due to quantization of 
BDCT coefficients, C and C 1 respectively, the block based forward and inverse DCT matri- 
ces and Q and D 1 , respectively, the corresponding quantization and the inverse quantization 
matrices The restoration phase represents the post filtering process for reducing block ar- 
tifacts Based on the image representation model given m eqn (2 2), the above mentioned 
matn es can be represented as 
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(2 4) 


where [c*] and [c*] -1 , respectively represents the forward and inverse DCT matrices, with 
size B 2 xB 2 , for processing lexicographically ordered BxB image blocks The quantization 
operator is further divided into two successive operations, division and rounding off, such os 


Q = RD 

In eqn (2 5), the division matrix D can be written as 


[dj 0 0 

0 [da] 0 

D= 0 0 

0 0 

o [dtfa/B 2 ] 
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(2 5) 


(2 6 ) 


where the kth diagonal submatnx [d k ] is a diagonal matrix whose diagonal elements are 
determined by the following way For a JPEG based quantization table as shown m fig 2 2 


j=l,j=2, , j=8 

i=l 

i=2 


i=8 

Figure 2 2 A quantization table used for 8x8 DCT coefficients 

diagonal elements of [dk] can be represented as 

d k {l,l) = — for I = (» — 1) 8 + 1<M<8 (2 7) 

where represents the (t,j)th value m quantization table Rounding operation R 

can be written as a diagonal matrix whose diagonal elements performs rounding operation 
to the corresponding elements of input vector The inverse quantization matrix, denoted by 
.D -1 , simply represents the inverse of the division matrix D 
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Chapter 3 


Proposed Adaptive Image Restoration 


Based on the degradation model descnbed in the previous section an adaptive image 
restoration has been formulated In subsequent sections restoration algorithm and an edge 
classification algorithm using DCT coefficients is proposed 

3 1 Proposed Image Restoration Algorithm 

Equation (2 2) can be written as 

y = C-'D-'QCx = Ex (3 1) 

where H can be considered as an image degration operator Many BDCT based moving 
image compression technique, such as H 261, MPEG 1 and MPEG 2 have the same block 
artifacts when the bit rate becomes less than the certain threshold value It is known that 
moving images have larger application areas than still images and block artifacts in moving 
images may degrade the quality of images more severely because block artifacts appear at 
the instance of rapid scene change For removing block artifacts m motion images, the 
iterative or optimization based type restoration algorithm are not suitable due to their slow 
convergence Even in still image application, slow convergence is definitely a disadvantage 

For this reason a fast image restoration algorithm using classification based constrained 
optimization has been proposed for reducing block artifacts m BDCT-based compressed 
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images A general image restoration process based on the constrained optimization approach 
is to find x which satisfies 

\\y- Hx\\ = 0 (3 2) 

subject to 

||Ax|| 2 < e 2 (3 3) 

where A represents a high pass filter and inequality equation(3 3) guarantees that high 
frequency component m the solution is controlled below the pre-specified quaantity The 
functional m eqn (3 2) represents tiie energy of the residual The original undegraded im- 
age x , the reconstructed image y with block artifacts and many other images can be el 
ements of the solution set defined by eqn (3 2) One way to select a solution is to keep 
variance across the block boundary less than a certain value by appropriately choosing a 
space-variant high pass-filter in eqn (3 3) The degradation operator H m the BDCT-based 
compression reconstruction process is nonlinear and space-vanant due to their nature of the 
rounding matrix Therefore an approximated version of the constrained optimization has 
been proposed, which minimizes 

III/ -IMP (3 4) 

with respect to x, subject to 

||A B x|| 2 <e 2 (3 5) 

Here Hi is assumed as a space invariant low-pass filter and Ae is assumed as a block- 
adaptive directional high pass filter whose direction is determined by block-classified edge 
information DCT coefficients with higher frequency components tend to be more coarsely 
quantized Therefore H is considered as a low pass filter Hi Minimization of eqn (3 2) can 
suppress the smoothing operation on non boundary region in the image A E smoothes the 
block boundaries while preserving directional edges which are not the results of the degra- 
dation process of the block coding The detailed process of block-based edge classification 
technique is descnbed in the following section 
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3 2 Blocked-based Edge Classification 


A block-classification technique using a part of DCT coefficients has been described which 
is used for implementing the block-adaptive directional high-pass filter m equation(3 5) For 
a BxB image block, the corresponding DCT coefficients are expressed as[12] 


B—l B - 1 


C x (ki, k\) — c(ki) c(k 2 ) 53 %(ni,n2)cos~ki(2ni + 1) x cos-^=k 2 {2n 2 + 1), (3 6) 
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where 


c(k) = 


yj for k=0, 

^ -/f for k=l,2, B-l 


(3 7) 


For block size of 8x8, only (7(0, l) and (7(1,0) have been used out of sixty four DCT 
coefficients which represent vertical and horizontal edges respectively^] These are given as 

a /2 7 7 IT 

C veT = C( 0, 1) = — 53 53 x(ni,n 2 )cos——2(n 2 + 1), (3 8) 

0 m=0n 2 =0 ZXB 

a /2 77 trr 

Chor = (7(1,0) = — 53 53 x(m,n 2 )cos — — 2(n x + 1) (3 9) 

8 m =0*12=0 2x8 

Using above two equations the directions of the edges in each block has been determined 
The edges of the 8x8 block is classified into monotone, horizontal, vertical, 0° ~ 45°, 45° ~ 
90°, 90° 135°, and 135° ~ 180° edges The edge classification algonthm[4] is given in the 

following 

Algorithm 1 (Edge classification) 

If \C ver \ \Chor\ < o 2 , then the block is monotone, 

else if \C veT \ > a and \Ch or \ < then vertical edge exists, 

else if \C ver \ < a and |(7/,or| > a, then horizontal edge exists, 

else if C ver Chor > e 2 and \C ver \ > C hor , then 0° ~ 45° edge exists 

else if C ver 0^ > a 2 and \C hor \ > C ver , then 45° 90° edge exists, 

else if C ver Chor < a 2 and \C hoT \ > C veT then 90° ~ 135° edge exists, 

else 135° ~ 180° edge exists 

Here cr is a positive quantity(0 001 ~ 25 0) which controls the sensitive of edge classification 
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Chapter 4 


FIR Implementation For Real-time 
Processing 


According to image degradation model descnned earlier, the degradation operator H 
results in both discontinuities on block boundaries and loss of high frequency details inside 
blocks Among many practical applications of image restoration, the constrained least squares 
(CLS) restoration filter is widely used because it can incorporate a pnon constraints into 
the restoration process The frequency response of a typical CLS restoration filter is given 
by [4] 

1 |ff(M)l 2 + A|<4(M)l 2 1 J 

where H ( k , l ) and A(k, l ) respectively represent the two-dimensional (2D) Discrete Fourier 
Transform (DFT) of the degradation operator and a high pass filter In the denominator 
of G(k,l) in equation (4 1), \H(k,l)\ 2 tnes to reduce the degradation due to H, \A(k, l)\ 2 
suppresses excess amplification of high-frequency components and the Lagrange multiplier 
A is determined such that the solution satisfies the equality in eqn (3 3) subject to eqn (4 1) 
As discussed earlier degradation due to quantization of BDCT coefficients is neither linear 
nor space-invanant and as a result, the CLS restoration filter cannot be implemented This 
is why we apply low pass filtering inside blocks and high pass filtering on block boundaries 
The first low pass filtering function is modeled by an approximated linear space-invariant 
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filter whose point spread function (PSF) is given as [4] 


hL{m, n) 


22 


1 2 1 
2 10 2 
1 2 1 


(4 2) 


For a specific block with a certain edge type, denoted by the subscript E, the low-pass 
filtering degradation can be restored by the following approximated CLS restoration filter 


G E {k, l ) 


(4 3) 


\H L (k,l)\^ + X\A E (k,l)r 

where and A E (k,l) respectively represent the 2D DFT of h E (m,n) and the 

directional high-pass filter a E (m, n), E e [mono, ver, hor, 45°, 135°] with proper zero padding 
[7,10] to make it 8x8 The point spread functions for different high pass filtering are given 
as[4] 
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ahor(m, n ) 
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The low pass filter hi in eqn (4 2) and each high-pass filter a# axe empencally chosen 
by considering the fact that the region of support should become as small as possible with 
satisfactory restoration performance In implementing Gi(k, l) m eqn (4 3), the lagrange 
multiplier A =150 has been used for monotone edge block and A =75 has been used for the 
rest 

By taking the inverse DFT (IDFT) of G£(k,l) we obtain the coresponding spatial domain 
PSF gE(m,n) with support BxB Then the PSF is truncated by the 2D raised cosine of size 
MxM, M<N[8] 


w[m,ri 


stn§(m - ^pl) 

W - W) 


cos§ij)(m- ^1) 

" l ) 

cosjjiJ)(n — ^j-1) 

1 -$(ro «-»)» 

i" 

1 

St, 

< 

(4 9) 


where ip ( 0 ~ 1) is the roll off factor Value of M is such that the size of the resultmg filter 
is minimized with proper bandpass characterstics maintained The procedure for obtaining 
adaptive restoration filter is explained m fig 4 1 


An adaptive restoration filter m an arbitrary direction is obtained by using five different 
gE(m,ri) such as [4] 

g(m, n) = a u(m, n) + (1 - a) {(3 g x {m, n) + (1 - /?) g 2 (m, n)}, (4 10) 

where u(m,n ) represents the 2D impulse /? 6 [0,1] is the weight between gi(m, n) and 
g 2 (m,n),a is the scaling factor which normalizes g(m,n ) and g x and g 2 are determined by 
the following algorithm [4] 

Algorithm 2 ( Choice of two directional restoration filters) 

According to the edge classification algorithm given m Algorithm 1 
(i) 9i = 92 = 9 mono> for monotone edge 

(u) 9i = 92 = 9ver, for vertical edge 
(in) g x = g 2 = g hor , for horizontal edge 

(iv) g x = g ver and g 2 = 545 , for 0° ~ 45° edge 

( v ) 9i = 545 and g 2 = g ho r, for 45° ~ 90° edge 
(vi) gx = g h0T and g 2 = 5135 , for 90° ~ 135° edge 
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Figure 4 1 Procedure for obtaining five different FIR restoration filters, where blocks denoted 
by ”T” represent truncation by the proper size of raised-cosine window 
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C ver ^hor 


Figure 4 2 Proceduie for implementing the adaptive FIR restoration filter for each block 

(vn) gi =gi 35 and g 2 = g ver , for 135° ~ 180° edge 

In equation (4 9), (3 depends on the value of C ver and and a depends on the (l,2)th 
and (2,l)th value m the quantization table as shown m fig 2 2 

The procedure of implementing the adaptive FIR restoration filter for each block is shown 
m fig 4 2 

It is to be noted here that ^(m n), E e [memo, ver, hor, 45°, 135°] can be obtained 
well before the restoration process starts In processmg each block g(m,n) for each block is 
computed by using the above two algorithms and the convolution of the corresponding block 
with the g(m,n ) is performed 
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Chapter 5 


Simulation Results 


The two algorithms described earlier have been developed using C programming Images 
of 256x256 pixel values have been considered and each image is again subdivided into no of 
blocks of size 8x8 The 256x256 Lena image has been compressed and then reconstructed by 
computer simulation according to the degradation model shown m fig 2 1 In the degradation 
prpcess, a JPEG quantization table has been used as shown earlier m fig 2 2 Original and 
the compressed images have been shown m figures 5 1, 5 2 and 5 6 using two different 
quantization factors, Q 1 and 1 5 Different edges have been classified using algorithm 1 and 
five different restoration filters m different directions have been developed by using algonthm- 
2 Lena image has been processed by those five different restoration filters The resulting 
block artifacts reduced Lena images have been shown in figures 5 3, 5 4, 5 5, 5 7, 5 8, 5 9 
using two different quantization factors, Q When the subjective quality is compared it is 
evident that the restored image by using five different restoration directional filters is the 
winner in the sense of both reducing block artifacts and preserving edge details inside each 
block For each quantization factor, Q three different restored Lena images have been shown 
for three different values of scaling factor a (0 4, 0 6, 0 75) used m the equation(4 10) It 
has been observed that removing the blocking artifacts completely results m the increase in 
blurring m the restored images Considering this fact a compromise has to be made between 
blocking artifacts and blurring m the pictures 

Another image of Claire (fig 5 10) of size 256x256 has been considered and the same has been 
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Figure 5 1 Original Lena image 


compressed and processed for the same quantization factor, Q and for three different values 
of scaling factor a as mentioned earlier using algorithm 1 and algorithm 2 Corresponding 
to those different values of Q and a compressed and processed Claire images also have been 
shown in figures 5 11, 5 12, 5 13, 5 14, 5 15, 5 16, 5 17, 5 18 
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Figure 5.2: Compressed Lena image with quantization factor, Q=1 



Figure 5.3: Processed Lena image with Q=1 and u=0A 
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Figure 5.4: Processed Lena image with Q=1 and «=0.6 
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Figure 5.6: Compressed Lena image with quantization factor, Q=l. 5 



Figure 5.7: Processed Lena image with Q=1.5 and a=0A 
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Figure 5.8: Processed Lena image with Q=1.5 and o:=0.6 



Figure 5.9: Processed Lena image with Q=1.5 and a=0.75 
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Figure 5.10: Original Claire image 
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Figure 5.12: Processed Claire image with Q=1 and a=0.4 



Figure 5.13: Processed Claire image with Q=1 and a=0,6 
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Figure 5.14: Processed Claire image with Q=1 and a= 0.75 



Figure 5.15: Compressed Claire image with quantization factor Q — 1.5 
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Figure 5.16: Processed Claire image with Q=1.5 and a=0A 



Figure 5.17: Processed Claire image with Q=1.5 and o;=0.6 
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Figure 5.18: Processed Claire image with Q=1.5 and a=0.75 
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Chapter 6 


Conclusion 


In this thesis an adaptive image restoration filter for reducing block artifacts has been 
developed m BDCT based compressed images Although the proposed restoration filter can 
efficiently reduce the block artifacts, the quality of the processed image is not as good as 
that of the original one due to missing information in the quantization process and the 
inherent low-pass characteristics m the proposed filter The major contribution of this thesis 
is that it enables real time constrained optimization processing for reducing block artifacts 
by using simple edge classification technique In comparing the proposed algorithm with 
conventional iterative type algorithms, which are used mainly for still images, there is no 
significant degradation m the proposed image while the computation time is greatly reduced 

6 1 Scope for Future Work 

In this thesis the whole work has been done with achromatic images only This can be 
extended to color images also Though the work has been done with stored still images 
only, this work can be carried out m real-time processing also The algorithms used here 
can be used in many applications like edge detection in JPEG, MPEG, etc The edge 
detection process described earlier can be extended to extract more complex feature of the 
still images and moving images The proposed filter may be used as a postprocessor in 
the decoder of moving image compression systems, such as digital VCR, video on demand, 
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digital HDTV systems, medical processing etc Some works can be carried out to find more 
effective algorithms which will work adaptively according to the quantization matrix and the 
characteristics of the source lamge 
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